Incremental Sensorimotor Learning with Constant Update Complexity

The field of robotics is increasingly moving toward applications that involve unstructured human environments. This domain is challenging from a learning perspective, since subsequent observations are dependent and the environment is typically non-stationary. This non-stationarity is not limited to the external environment, as internal sensorimotor relationships may be subject to change as well. Specific causes include environmental influences (e.g., temperature) and wear-and-tear, as well as self-initiated changes (e.g., changing dynamics due to tool-use). Successful modeling of sensorimotor relationships therefore necessitates an open-ended learning process that continuously updates existing models when novel observations become available. Adaptation and prediction have to take place while the robot is operating in the environment and should be time-efficient to ensure responsive behavior. The implied requirements of incremental updates and O(1) persample time complexity, however, are not well supported by current machine learning methods. For instance, kernel methods are “impaired” by formulating the solution in terms of a kernel expansion that grows linearly with the number of training samples. Even algorithms targeted specifically for real-time, incremental robotic learning, such as LWPR (Vijayakumar, D’souza, and Schaal 2005), are characterized by steadily increasing computational requirements as the number of training samples grows ad infinitum. Furthermore, this latter method generally requires a large amount of training samples and manual hyperparameter tuning to attain satisfactory performance.